Comparison of Different Estimation Methods for Linear Mixed Models and Generalized Linear Mixed Models

Linear mixed models (LMM) and generalized linear mixed models (GLMM) are widely used in regression analyses. With the variance structure dependent on the random effects with their variance components, the parameter estimation of LMMs is more complicated than linear models (LM). Generally, we use maximum likelihood estimation (MLE) together with some procedure such as derivative free optimization to commit the estimation of LMMs. For GLMMs, the computation is even more challenging due to the high-dimensional integration for the marginal likelihood. Classical literature on fitting GLMM are generally based on some Laplace-type approximation, but the estimators are asymptotically biased. The recent studies of GLMMs estimation are mostly focused on the computer-based Markov Chain Monte Carlo (MCMC) method, which is believed to obtain more accuracy on variance estimates. This paper intends to study some mainly used likelihood approximation methods such as Laplace approximation, Gauss-Hermite quadrature (GHQ), and penalized quasi-likelihood, as well as MCMC methods. These methods are applied on two classical data sets to make comparisons between the estimation methods. The computations are supported by R and OpenBUGS.